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Background: This study describes a repeated measures prediction index to identify patients at high risk of > grade 2 
hand-foot skin reaction (HFSR) before each week of sorafenib therapy. 

Methods: Data from 451 patients who received a sorafenib (400 mg bid) as part of a clinical trial were reviewed 
(Escudier B, Eisen T, Stadler WM et al. Sorafenib in advanced clear-cell renal-cell carcinoma. N Engl J Med 2007; 356: 
125-134). Generalized estimating equations were used to develop the final risk model. A risk-scoring algorithm (range 
0-58) was then derived from the final model coefficients. External validation was then carried out on a new sample of 
1 145 patients who received sorafenib under an expanded access program. 

Results: Pretreatment white blood cell count, female gender, good performance status, presence of lung and liver 
metastases and number of affected organs were predictors for > grade 2 HFSR. A nonlinear association between 
HFSR risk and treatment duration was also identified where risk was maximized at week 5 followed by a gradual 
decline. Before each week of therapy, patients with risk scores >40 would be considered at high risk for developing > 
grade 2 HFSR. 

Conclusions: The application and planned continued refinement of this prediction tool will be an important source of 
patient-specific risk information for the development of moderate to severe HFSR. 
Keywords: hand-foot skin reaction, prediction, renal cell carcinoma, risk, sorafenib 



introduction 

Sorafenib is an orally administered inhibitor of several tyrosine 
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95% confidence interval (CI) 3.9-14.4] [8]. Prospective 
evaluations of patients on sorafenib showed that HFSR 
developed in 60% of patients, with 23% being grade >2 in 
severity [9]. 

Grade >2 HFSR is of clinical concern because it can lead 
treatment delays, dose reductions and/or premature treatment 
discontinuation of sorafenib, even in situations where other 
toxic effects may be manageable and the disease is responding 
to therapy. Even though risk factors for HFSR such as female 
gender and patient performance status have been identified, 
they have not been specific to sorafenib [10, 11]. Therefore, 
occurrences of HFSR during sorafenib therapy are largely 
considered to be unpredictable. As a result, oncologists 
generally take action only after the event occurs (i.e. reactively). 
This involves trying to 'rescue' the patient from a subjectively 
unpleasant situation and then making adjustments to the 
regimen such as dose reductions, delays and/or the initiation of 
supportive care interventions. Clinical care could be 
substantially improved and dose intensity could be maintained 
if these episodes of significant HFSR could be accurately 
predicted, with steps taken in advance to prevent their 
occurrence in the first place (i.e. proactively). Such steps might 
include the use of appropriate supportive care medication as 



protein kinases, including VEGFR, PDGFR and Raf [1]. It has 
been approved by regulatory agencies for use in patients with 
advanced renal cell (RCC) and hepatocellular carcinoma 
(HCC) following the results of randomized trials that 
demonstrated an improvement in progression-free and overall 
survival [2-4]. As with many other targeted therapies, 
sorafenib is associated with dose-limiting adverse events (AEs) 
[5]. Among these, hand-food skin reaction (HFSR) is 
noteworthy due to a painful erythema, edema and 
desquamation of the palms and soles, which can lead to 
decreased quality of life [6, 7]. In the pivotal randomized trial 
in advanced RCC, the incidence of grade >2 HFSR was ~18% 
[2]. In a recent meta-analysis of phase II and III studies and 
expanded access programs of patients with solid tumors, the 
risk of HFSR in patients receiving sorafenib was increased 
more than seven times relative to control [relative risk = 7.50, 
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well as forewarning the patient and initiating a more intensive 
early monitoring scheme and action plan for early 
intervention. 

The realities of health care systems around the world 
preclude such arrangements being put in place, for all patients, 
throughout all cycles of sorafenib therapy. What may be 
possible, however, is a highly focused strategy based on the 
accurate prediction of patients at higher than average risk, 
applied 'just-in-time' to preempt episodes of grade >2 HFSR. 
In other words, it should be possible and economical to 
intervene preventatively if we knew who was at higher risk and 
when (i.e. at what point in time) the risk would become 
unacceptably high. These goals could be achieved through the 
development and validation of an HFSR prediction tool that 
could be applied at multiple time points (i.e. repeated 
measures). 

The risk of HFSR during sorafenib therapy may vary over 
time. This has been the case with other toxic effects such as 
neutropenia and emesis [12, 13]. Therefore, the advantage of a 
repeated measures compared with a pretreatment only tool is 
that the former allows risk to be continually reassessed with 
each additional week of sorafenib therapy. Repeated measures 
models have been successfully developed for other toxic effects 
such as chemotherapy induced emesis and neutropenia [13, 
14]. In this study, the development of a repeated measures (i.e. 
by week) prediction index for grade >2 HFSR in patients 
receiving sorafenib is described. 

materials and methods 

patients and treatment 

Patient data for developing the repeated measures risk index for HFSR were 
obtained from the study reported by Escudier et al. [2], a large randomized 
placebo controlled trial evaluating sorafenib in patients with advanced 
RCC. In this clinical trial, a total of 903 advanced- stage RCC patients who 
were previously exposed to cytokines were randomized to receive oral 
sorafenib 400 mg twice daily (n = 451) or to placebo. Baseline data 
collection included patient demographic information, Eastern Cooperative 
Oncology Group performance status, biochemistry, number and site of 
metastases, median duration of disease and previous therapies. Over the 
first 8 weeks of sorafenib therapy, 43 patients developed > grade 2 HFSR. 
By the end of the study, the median duration of sorafenib was 23 weeks 
with 80 of 451 (17.7%) patients developing > grade 2 HFSR as defined by 
the National Cancer Institute-Common Toxicity Criteria (NCI-CTC). 
Therefore, the primary risk model end point was grade >2 HFSR as 
defined by the NCIC-CTC over the first 8 weeks of therapy. 

predictive factors for HFSR and development of 
scoring index 

There is evidence to suggest that sorafenib induced HFSR is most prevalent 
within the first 4 weeks of therapy [15]. Consequently, clinical outcomes 
data for the first 8 weeks of sorafenib therapy were used to build the risk 
model and associated scoring index. For the first 8 weeks of sorafenib 
therapy, the database from the Escudier et al., trial provided 3608 weeks of 
complete outcomes data. Patient demographic and clinical variables were 
screened for possible inclusion into the risk model. To identify the set of 
factors with the largest potential contribution to HFSR risk, those with a 
P value <0.25 in a simple logistic regression with the dependent variable of 
HFSR were retained for further consideration. This is a recommended 
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approach for removing weak prognostic covariates so that a more 
manageable set of variables can be submitted to multivariate 
techniques [16]. 

To determine the final set of risk factors, generalized estimating 
equations (GEE) were used, which adjusts for the clustering by week of 
therapy within a patient [17, 18]. A GEE model was chosen since multiple 
weekly observations for the same patient would be expected to violate the 
independence assumption of standard logistical regression. The set of 
initially retained risk factors was analyzed in the GEE model. The 
likelihood ratio test was then used in a backwards elimination process 
(P < 0.05 to retain) to select the final set of risk factors for retention into 
the model. To evaluate the presence of a nonlinear relationship between 
HFSR risk and week of therapy, the quadratic form of time (i.e. time 2 ) was 
assessed in the model. The goodness of fit of the final model was then 
assessed with the Hosmer-Lemeshow test. Model calibration was evaluated 
by estimating a smooth calibration line between the observed and predicted 
outcomes [19]. The calibration curve would equal one (optimal) if the 
observed and predicted probabilities agree perfectly. 

Nonparametric bootstrapping was applied to test the internal validity of 
the final prediction model [20, 21]. Resampled data (1000 iterations) were 
used to generate bootstrap estimates of the regression coefficients of the 
multivariable model. The CIs of the regression coefficient estimates from 
the bootstrap sampling were then compared with the values calculated by 
the GEE regression analysis. 

From the GEE regression model, the contribution of the individual risk 
factors to HFSR risk was weighted with the model coefficients. To simplify 
calculations using these weights in the risk algorithm, the coefficients were 
transformed by multiplying each by a constant (derived by trial and error) 
and then rounding to the nearest unit value. A summary of HFSR risk 
score was assigned to each patient by simply adding up transformed 
coefficient values ( points) for each risk factor they possessed. 

external validation of HFSR prediction index 

The external validation sample consisted of 1145 patients (a mix of 
advanced- stage RCC and HCC) who received sorafenib as part of a 
European expanded access program (i.e. EU-ARCCS) [22]. This database 
provided 9160 weeks of complete outcomes data. However, it is important 
to note that since this data were not derived from a clinical trial, there were 
some baseline variables such as hemoglobin (Hb) and white blood cell 
(WBC) data that were not collected. 

The predictive accuracy of the final risk scoring index was then 
determined by measuring the specificity, sensitivity and area under the 
receiver operating characteristic (ROC) curve [23, 24]. Discrimination 
refers to the ability of a diagnostic test or predictive tool to accurately 
identify patients at low and high risk for the event under investigation and 
is often presented as the area under the ROC curve. A predictive 
instrument with an ROC of >0.70 is considered to have good 
discrimination, and an area of 0.5 is equivalent to a 'coin toss'. All the 
statistical analyses were carried out using Stata, VI 1.0 (Stata Corp., College 
Station, TX). 

results 

The clinical and disease characteristics of patients in the model 
derivation and validation samples are presented in Table 1 . 
Patients in both groups were comparable with respect to mean 
age, gender and presence of lung and liver metastases. 
However, there were a higher proportion of patients in the 
derivation dataset with a performance status of zero or one and 
with multiple organ involvement. Notwithstanding, it is 
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Table 1. Characteristics of patients in the derivation and validations samples 



Characteristic 

Mean age (range) 
Female gender 

ECOG performance status (%) 
0 
1 
2 

Mean baseline Hb (g/dl) (SD) 
Mean baseline WBC (xlO 9 cells/1) 
Median number of organs involved (range) 
Multiple (more than one) organ involvement 
Presence of liver metastases 
Presence of lung metastases 
Development of grade >2 HFSR 



Derivation sample (n = 451) a 

59.1 (19-86) 
30.2% 

48.7 
49.8 

1.6 
12.8 (2.1) 

7.4 (2.8) 
3 (1-3) 
85.8% 
79.6% 
25.7% 

43 affected patientsover 8 weeks (9.5%) 



External validationsample (n = 1145) a 

60.9 (18-84) 
25.3% 

40.1 

45.2 
14.7 
N/A 
N/A 

2 (1-3) 
76.1% 
73.0% 
27.7% 

220 affected patientsover 8 weeks (19.2%) 



a The model derivation sample consisted of RCC patients treated within a clinical trial. In contrast, the external validation sample consisted of both RCC and 
hepatocellular carcinoma who received sorafenib under an expanded access program. 

ECOG, Eastern Cooperative Oncology Group; Hb, hemoglobin; HFSR, hand-foot skin reaction; N/A, data not available; RCC, renal cell carcinoma; SD, 
standard deviation; WBC, white blood cell. 



Table 2. Final HFSR prediction model developed from the derivation dataset 



Variable 3 

Female gender 
ECOG PS >1 

Lung metastases at baseline 
Liver metastases at baseline 
2 or more organs involved 
Baseline WBC >5.5 (xlO 9 cells/1) 
Time (week of therapy) 
Time 2 



Odds ratio (95% CI) b 

1.68 (1.30-2.18) 
0.50 (0.38-0.65) 
0.58 (0.43-0.79) 
1.75 (1.33-2.29) 
1.71 (1.10-2.66) 
1.56 (1.15-2.09) 
1.54 (1.18-2.0) 
-0.96 (0.93-0.99) 



npact on HFSR relative risk 

Increased by 68% 
Decreased by 50% 
Decreased by 42% 
Increased by 75% 
Increased by 71% 
Increased by 56% 

Increased by 54% per week of therapy 
Nonlinear association between risk and time 



a These are the final variables that were retained following the application of the likelihood ratio test (P < 0.05 to retain) in a backwards elimination process. 
b 95% CI determined by nonparametric bootstrapping. 

CI, confidence interval; ECOG PS, Eastern Oncology Cooperative Oncology Group performance status; HFSR, hand-foot skin reaction; WBC, white blood 
cell. 



important to recall that this is not a randomized trial but an 
exercise to develop an HFSR prediction model from unique 
patient samples. Therefore, imbalance between model 
derivation and validation samples should be expected and even 
encouraged to ensure that the prediction model can be applied 
to a variety of patients being treated with sorafenib. 



therapy was also identified where risk was maximized at week 
5 followed by a gradual decline (Figure 1). The CIs of 
regression coefficient estimates from the bootstrap sampling 
were comparable with the values calculated by the GEE 
regression analysis, supporting the internal validity of the 
model. 



development of HFSR risk prediction model 

After the initial univariate screening of potential predictor 
factors, the variables associated with HFSR retained for further 
analysis were gender, age, performance status, baseline Hb and 
WBC count, presence of liver and lung metastases and number 
of organs involved. The development of the prediction model 
was continued with multivariable GEE regression analysis and 
the backwards elimination process. The final variables retained 
in the model that were significant predictive factors (P < 0.05) 
for HFSR were female gender, patient performance status of 
one or two, presence of liver and lung metastases, two or more 
organs involved and a normal baseline WBC count (Table 2). 
A nonlinear association between HFSR risk and duration of 



development of HFSR scoring system 

A risk scoring system was then developed from the point 
estimates of the regression coefficients and the intercept 
generated from the analysis. Each of the final regression 
coefficients retained in the model provided a statistical weight 
for that factor's contribution to the overall risk of HFSR. The 
scoring system was then adjusted by adding a constant across 
all scores to ensure that none were <0. The final product was a 
scoring system between 0 and 58 where higher scores were 
associated with an increased risk for a HFSR event (Table 3). 

Factors that elevate the overall score are considered to be 
positive risk factors for HFSR. For instance, female gender 
requires the addition of six units and is thus a risk factor for 
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Figure 1. Probability of developing hand-foot skin reaction for a female 
renal cell carcinoma patient, ECOG 1, liver metastases only and white 
blood cell 7.0. ECOG, Eastern Oncology Cooperative Oncology Group. 



Table 3. Risk scoring system for > grade 2 HFSR in patients receiving 
sorafenib 



Predictive factor 


Start sorafenib 


Initial starting score 


Initial score = 20 


Impact of baseline factors 




Female gender 


Add 6 


ECOG PS >1 


Subtract 7 


Lung metastases at baseline 


Subtract 7 


Liver metastases at baseline 


Add 6 


2 or more organs involved 


Add 9 


Baseline WBC >5.5 


Add 5 


Impact of HFSR risk by week of therapy 




Week 1 


Add 4 


Week 2 


Add 7 


Week 3 


Add 10 


Week 4 


Add 11 


Week 5 


Add 12 


Week 6 


Add 11 


Week 7 


Add 10 


Week 8 


Add 8 


Total composite risk score 3 


? 



a The probability of developing HFSR during that given week while on 
sorafenib therapy can then be estimated from supplemental Figure SI or 
supplemental Table SI (available at Annals of Oncology online). 
ECOG PS, Eastern Oncology Cooperative Oncology Group Performance 
Status; HFSR, hand-foot skin reaction; WBC, white blood cell. 

the development of HFSR. In contrast, patients with a 
performance status of one or two are at a reduced risk for 
HFSR, so such patients would have seven units subtracted from 
their cumulative risk score. This risk scoring system can then 
be applied to an individual patient before the start of sorafenib 
and then weekly in order to monitor the risk of HFSR 
throughout the entire course of therapy. For illustration, a 
60-year-old RCC patient (base score = 20) who is female (add 
6) with a baseline performance status of one (subtract 7), who 
has liver metastases only (add 6) and a normal WBC (add 3) 
would have a risk score of 34 before starting sorafenib therapy. 
This would correspond to a model estimated-HFSR risk of 



~7% during the first week of therapy. As sorafenib is 
continued, the HFSR risk would increase until week 5 and then 
begin to decline (Figure 1). Therefore, the data would suggest 
that the most critical period of HFSR development is during 
the first 5 weeks of sorafenib. 

All patients in the derivation sample were assigned a risk 
score based on the above system. The risk score for each 
patient was then compared with the corresponding probability 
of developing a HFSR, as predicted from the final GEE 
regression model. The data suggested a nonlinear relationship 
between risk score and probability of HFSR (supplemental 
Figure SI, available at Annals of Oncology online). The model 
development was continued with an ROC curve analysis and a 
measurement of the area under the ROC curve on both the 
derivation and external validation datasets. The findings 
suggested that the area under the ROC curve in both the 
derivation and external validation samples was acceptable; 0.67 
(95% CI 0.64-0.70) versus 0.60 (95% CI 0.58-0.62), supporting 
the internal and external validity of the scoring system. 

The final step in the development of the prediction tool was 
the identification of a high-risk score threshold or 'cut-off, 
which optimized sensitivity and specificity and was able to 
minimize the misclassification rate. Five risk score categories 
were developed (supplemental Table SI, available at Annals of 
Oncology online). The analysis identified a risk score threshold 
of >40 as being the range where sensitivity and specificity are 
optimal and a high proportion (77.9%) of patients were 
correctly classified (supplemental Table SI, available at Annals 
of Oncology online). Using a risk score threshold of >40 would 
capture patients with a HFSR risk of ~15% before each week of 
sorafenib therapy (supplemental Figure SI, available at Annals 
of Oncology online). 

discussion 

This study describes the development and external validation of 
a predictive index designed to estimate the absolute risk of HFSR 
in patients receiving sorafenib therapy. From a list of potential 
predictor variables, the final risk model contained seven 
variables that were retained by the statistical elimination process. 
The finding that females and good performance status were 
significant independent risk factors for HFSR was consistent 
with the literature [9, 10]. In addition, the analysis revealed the 
nonstatic nature of HFSR toxicity whereby the incidence 
increased by week 5 followed by a gradual decrease (Figure 1). 
This suggests that risk increases by cumulative drug exposure 
with the first few weeks being the most critical [15, 25]. 

Sorafenib is primarily metabolized in the liver, undergoing 
oxidative metabolism, mediated by CYP3A4 as well as 
glucuronidation mediated by UGT1A9 [1]. Patients with a 
normal WBC at baseline would also more likely have a starting 
dose of 400 mg twice daily as opposed to a lower dose. 
Therefore, the findings that a patient with liver metastases and 
a normal WBC count are at a higher risk for HFSR are 
consistent with the cumulative drug exposure hypothesis. The 
observation that patients with lung metastases have a lower 
HFSR risk was unexpected and requires further investigation. 

The intent of the prediction exercise was to provide 
oncologists a tool that would enable them to estimate the 
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percent risk of HFSR prior each week of sorafenib exposure. 
To facilitate medical decision making, several cut points with 
risk thresholds were developed (supplemental Table SI, 
available at Annals of Oncology online). These risk score 
thresholds are not fixed and can vary based on the patient or 
oncologist's risk tolerance. Some people may prefer to select a 
lower risk threshold before the initiation of preventative 
interventions. A lower score threshold such as >30 would have 
a greater sensitivity (88.9%), which would minimize the false- 
negative rate (i.e. more people would receive preventative 
interventions who actually needed them). In contrast, the 
impact of using a lower threshold is a higher rate of false 
positives (i.e. more patients who do not really need the 
preventative interventions would receive them). Based on our 
suggested risk threshold, the hypothetical female RCC patient 
described earlier would be classified as low risk with an 
estimated-HFSR risk of ~7% before starting sorafenib. 
However, by the third week of sorafenib exposure, she would 
be considered at 'high risk' for HFSR according to the model 
because her score would be 41. 

Our study has some limitations. Our statistical power to 
identify important risk factors was limited with only 43 
patients developing > grade 2 HFSR in our model derivation 
sample. The predictive accuracy of our index was adequate 
with an area under the ROC curve of 0.67 suggesting that it 
was able to correctly discriminate two of every three exposed 
patients. However, there is room for improvement. WBC data 
were not available in the external validation sample. Therefore, 
the contribution of baseline WBC count to the overall risk 
calculation could not be evaluated during the external 
validation exercise. The model only considered data on readily 
measurable variables. Hence, not all the variability was 
accounted for in the analysis. Additional factors such as 
biological and molecular features of the tumors could be 
involved in the pathogenesis of HFSR. However, this data were 
not available for all patients. Future HFSR risk models should 
consider the inclusion of these factors in order to improve 
overall predictive accuracy. 

Despite these limitations, the final model allowed the 
development of a risk scoring index for identifying high-risk 
patients before each week of sorafenib exposure. The scoring 
index is easy to apply, able to reasonably discriminate between 
high- and low-risk patients and the threshold can be varied 
depending on a patient's risk tolerance. Therefore, the 
application and planned continued refinement of this 
prediction tool will be an important source of risk information 
for the practicing oncologist and it could enhance patient care 
by preempting the development of moderate to severe HFSR. 
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Background: Retrospective and molecular biologic data suggest that sunitinib may be effective in patients with non- 
clear cell renal cell carcinoma (nccRCC). 

Patients and methods: Eligibility criteria included advanced nccRCC except for collecting duct carcinoma and 

sarcomatoid carcinoma without identifiable renal cell carcinoma subtypes. Patients were treated with 50 mg/day oral 

sunitinib for 4 weeks, followed by 2 weeks of rest. The primary end point was overall response rate (RR). 

Results: Thirty-one eligible patients were enrolled. Twenty-four patients (77%) had prior nephrectomy. By Memorial 

Sloan-Kettering Cancer Center criteria, 8 patients (26%) had poor risk and 14 (45%) had intermediate risk. Twenty-two 

patients had papillary renal cell carcinoma (RCC), and three had chromophobe RCC. Eleven patients had partial 

response with a RR of 36% (95% confidence interval (CI) 1 9% to 52%) and an additional 1 7 patients (55%) had stable 

disease. Median duration of response was 12.7 months (95% CI 6.3-19.1 months), and median progression-free 

survival was 6.4 months (95% CI 4.2-8.6 months). At a median follow-up duration of 18.7 months (95% CI 13.7- 

23.7 months), 13 patients (42%) had died, resulting in an estimated median survival of 25.6 months (95% CI 8.4-42.9 

months). Toxicity profiles were commensurate with prior reports. 

Conclusions: Sunitinib has promising activity in patients with nccRCC (NCT01219751). 

Key words: chromophobe type renal cell carcinoma, papillary type renal cell carcinoma, phase II, sunitinib 



introduction 

Renal cell carcinoma (RCC) is the most common cancer of the 
kidney [1]. Up to 30% of patients with RCC present with 
metastatic disease, and recurrence occurs in ~40 % of patients 
treated for localized tumors [2]. Although RCC constitutes 
<5 % of adult malignancies, it is the sixth leading cause of 
cancer deaths worldwide because of a lack of effective therapy 
for advanced disease [ 1 ] . 

RCCs are composed of several distinct subtypes, including 
clear (conventional), papillary (types 1 and 2), chromophobe, 
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collecting duct, medullary type, and unclassified [3], which 
differ in pathogenetic mechanisms as well as in histological 
appearance and clinical course. Clear cell type RCC is the 
predominant subtype, constituting ~75 % of renal epithelial 
tumors. The remaining histological subtypes are collectively 
identified as non-clear cell RCC (nccRCC). Except for 
collecting duct carcinoma, which is sensitive to platinum-based 
chemotherapy, metastatic nccRCC is resistant to both 
immunotherapy and cytotoxic chemotherapy, resulting in poor 
patient prognosis [4] . Among histological subtypes, papillary 
renal cell carcinoma ( pRCC) has the poorest prognosis, with 
patients receiving cytokine therapy having a median overall 
survival (OS) of 6-8 months [5, 6]. 

The introduction of vascular endothelial growth factor 
(VEGF)-pathway inhibitors (sunitinib, sorafenib, pazopanib, and 
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